15 research outputs found
Benchmarking machine learning models on multi-centre eICU critical care dataset
Progress of machine learning in critical care has been difficult to track, in
part due to absence of public benchmarks. Other fields of research (such as
computer vision and natural language processing) have established various
competitions and public benchmarks. Recent availability of large clinical
datasets has enabled the possibility of establishing public benchmarks. Taking
advantage of this opportunity, we propose a public benchmark suite to address
four areas of critical care, namely mortality prediction, estimation of length
of stay, patient phenotyping and risk of decompensation. We define each task
and compare the performance of both clinical models as well as baseline and
deep learning models using eICU critical care dataset of around 73,000
patients. This is the first public benchmark on a multi-centre critical care
dataset, comparing the performance of clinical gold standard with our
predictive model. We also investigate the impact of numerical variables as well
as handling of categorical variables on each of the defined tasks. The source
code, detailing our methods and experiments is publicly available such that
anyone can replicate our results and build upon our work.Comment: Source code to replicate the results
https://github.com/mostafaalishahi/eICU_Benchmar
Domain-Aware Dialogue State Tracker for Multi-Domain Dialogue Systems
In task-oriented dialogue systems the dialogue state tracker (DST) component
is responsible for predicting the state of the dialogue based on the dialogue
history. Current DST approaches rely on a predefined domain ontology, a fact
that limits their effective usage for large scale conversational agents, where
the DST constantly needs to be interfaced with ever-increasing services and
APIs. Focused towards overcoming this drawback, we propose a domain-aware
dialogue state tracker, that is completely data-driven and it is modeled to
predict for dynamic service schemas. The proposed model utilizes domain and
slot information to extract both domain and slot specific representations for a
given dialogue, and then uses such representations to predict the values of the
corresponding slot. Integrating this mechanism with a pretrained language model
(i.e. BERT), our approach can effectively learn semantic relations
A Robust Data-Driven Approach for Dialogue State Tracking of Unseen Slot Values
A Dialogue State Tracker is a key component in dialogue systems which
estimates the beliefs of possible user goals at each dialogue turn. Deep
learning approaches using recurrent neural networks have shown state-of-the-art
performance for the task of dialogue state tracking. Generally, these
approaches assume a predefined candidate list and struggle to predict any new
dialogue state values that are not seen during training. This makes extending
the candidate list for a slot without model retaining infeasible and also has
limitations in modelling for low resource domains where training data for slot
values are expensive. In this paper, we propose a novel dialogue state tracker
based on copying mechanism that can effectively track such unseen slot values
without compromising performance on slot values seen during training. The
proposed model is also flexible in extending the candidate list without
requiring any retraining or change in the model. We evaluate the proposed model
on various benchmark datasets (DSTC2, DSTC3 and WoZ2.0) and show that our
approach, outperform other end-to-end data-driven approaches in tracking unseen
slot values and also provides significant advantages in modelling for DST
Scalable Neural Dialogue State Tracking
A Dialogue State Tracker (DST) is a key component in a dialogue system aiming
at estimating the beliefs of possible user goals at each dialogue turn. Most of
the current DST trackers make use of recurrent neural networks and are based on
complex architectures that manage several aspects of a dialogue, including the
user utterance, the system actions, and the slot-value pairs defined in a
domain ontology. However, the complexity of such neural architectures incurs
into a considerable latency in the dialogue state prediction, which limits the
deployments of the models in real-world applications, particularly when task
scalability (i.e. amount of slots) is a crucial factor. In this paper, we
propose an innovative neural model for dialogue state tracking, named Global
encoder and Slot-Attentive decoders (G-SAT), which can predict the dialogue
state with a very low latency time, while maintaining high-level performance.
We report experiments on three different languages (English, Italian, and
German) of the WoZ2.0 dataset, and show that the proposed approach provides
competitive advantages over state-of-art DST systems, both in terms of accuracy
and in terms of time complexity for predictions, being over 15 times faster
than the other systems.Comment: 8 pages, 3 figures, Accepted at ASRU 201
Whatās in a Food Name: Knowledge Induction from Gazetteers of Food Main Ingredient
We investigate head-noun identification in complex noun-compounds (e.g. table is the head-noun in three legs table with white marble top). The task is of high relevancy in several application scenarios, including utterance interpretation for dialogue systems, particularly in the context of e-commerce applications, where dozens of thousand of product descriptions for several domains and different languages have to be analyzed. We define guidelines for data annotation and propose a supervised neural model that is able to achieve 0.79 F1 on Italian food noun-compounds, which we consider an excellent result given both the minimal supervision required and the high linguistic complexity of the domain.Affrontiamo il problema di identificare head-noun in nomi composti complessi (ad esempio ātavoloā is the headnoun in ātavolo con tre gambe e piano in marmo biancoā). Il compito Ć© di alta rilevanza in numerosi contesti applicativi, inclusa lāinterpretazione di enunciati nei sistemi di dialogo, in particolare nelle applicazioni di e-commerce, dove decine di migliaia di descrizioni di prodotti per vari domini e lingue differenti devono essere analizzate. Proponiamo un modello neurale supervisionato che riesce a raggiungere lo 0.79 di F-measure, che consideriamo un risultato eccellente data la minima quantitĆ” di supervisione richiesta e la alta complessitĆ” linguistica del dominio
The Perfect Recipe: Add SUGAR, Add Data
We present the FBK participation at the EVALITA 2018 Shared Task ``SUGAR -- Spoken Utterances Guiding Chef's Assistant Robots''. There are two peculiar, and challenging, characteristics of the task: first, the amount of available training data is very limited; second, training consists of pairs \texttt{[audio-utterance, system-action]}, without any intermediate representation. Given the characteristics of the task, we experimented two different approaches: (i) design and implement a neural architecture that can use as less training data as possible, and (ii) use a state of art tagging system, and then augment the initial training set with synthetically generated data. In the paper we present the two approaches, and show the results obtained by their respective runs
FBKās Neural Machine Translation Systems for IWSLT 2016
In this paper, we describe FBKās neural machine translation (NMT) systems submitted at the International Workshop on Spoken Language Translation (IWSLT) 2016. The systems are based on the state-of-the-art NMT architecture that is equipped with a bi-directional encoder and an attention mechanism in the decoder. They leverage linguistic information such as lemmas and part-of-speech tags of the source words in the form of additional factors along with the words. We compare performances of word and subword NMT systems along with different optimizers. Further, we explore different ensemble techniques to leverage multiple models within the same and across different networks. Several reranking methods are also explored. Our submissions cover all directions of the MSLT task, as well as en-{de, fr} and {de, fr}-en directions of TED. Compared to previously published best results on the TED 2014 test set, our models achieve comparable results on en-de and surpass them on en-fr (+2 BLEU) and fr-en (+7.7 BLEU) language pairs
Proceedings of the Fifth Italian Conference on Computational Linguistics CLiC-it 2018
On behalf of the Program Committee, a very warm welcome to the Fifth Italian Conference on Computational Linguistics (CLiC-Āāit 2018). This edition of the conference is held in Torino. The conference is locally organised by the University of Torino and hosted into its prestigious main lecture hall āCavallerizza Realeā. The CLiC-Āāit conference series is an initiative of the Italian Association for Computational Linguistics (AILC) which, after five years of activity, has clearly established itself as the premier national forum for research and development in the fields of Computational Linguistics and Natural Language Processing, where leading researchers and practitioners from academia and industry meet to share their research results, experiences, and challenges